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An Expert System for Satellite and instrument 
Data Anomaly and Fault Isolation 

Carl Busse, Jet Propulsion Laboratory 
California Institute of Technology 

A prototype Generic Payload Operations Control System (GPOCC) 
is being developed at the NASA Jet Propulsion Laboratory to provide 
a low-cost command and control processing center for science 
instruments and small payloads. The GPOCC supports the difficult 
transition from integration and test to flight operations. The 
prototype will incorporate four expert systems to perform 
telemetry, command, and mission planning functions as well as 
telecommunications scheduling. The first of these expert systems 
to be developed will perform telemetry data analysis and fault 
isolation, as well as propose corrective action. 


This Data Analysis Module (DAM) will monitor telemetry data and 
perform continual data monitoring and trend analysis based on a 
knowledge base and historic data archived on an optical disk 
storage device. The system maintains a continuous "knowledge" 
database of past system performance characteristics. 


The Data Analysis Module will be partitioned into four stages: 

MONITORING - monitoring and interpreting instrument 
and satellite behavior. 

DIAGNOSIS - determine origin of system malfunctions 
inferred from knowledge base. 

PREDICTION - inference of predicted performance based 
on historic performance and current trends. 

RECOMMENDATION - developing and prescribing corrective 
action for diagnosed problems. 


The goal of the Data Analysis Module is to achieve consistent, 
dependable and validatable performance, to demonstrate thorough, 
reliable and fast reasoning, and to reduce the concentration 
demanded of flight analysis personnel. 


The work described in this paper was carried out by the Jet Propulsion Laboratory , California Institute 
of Technology, under a contract with the National Aeronautics and Space Administration. 
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Introduction 


The Data Analysis Module (DAM) is a project of the Institutional 
Computing and Mission Operations division of the California 
Institute of Technology, Jet Propulsion Laboratory. The Data 
Analysis Module is an expert system element intended for future 
incorporation in a purposed Generic Payload Operations Control 
Center (GPOCC) . The purpose of the DAM project is the design and 
development of an expert system to monitor spacecraft and science 
instrument telemetry data, isolate and diagnose faults, pinpoint 
the probable fault location and recommend corrective action. 


Background 

The Jet Propulsion Laboratory is a lead center for NASA's 
program of planetary exploration and earth science. In support of 
this role JPL has pursued areas of technology associated with the 
advancement of the spacecraft and science instrument operations 
environment. To concentrate emphasis efforts in new areas of 
technology the Laboratory established the Office of Space Science 
and Instruments (OSSI) and the Office of Technology and 
Applications Programs (TAP). [JPL Annual Report 1988] 


The concentration of the Laboratory on technology in the mission 
operations domain has centered on automation of instrument and 
spacecraft command and control functions. Automation in the 
mission environment is significant for three reasons. First, 
carefully selected automation reduces the workload of the 
operations personnel. Second, automated and verifiable computer- 
based tools [Harmod 1987] improve the accuracy of data processing and 
assist space flight and instrument control engineers in monitoring 
of spacecraft and mission sensors where operating data rates may 
greatly exceed the ability of individuals to monitor successfully. 
And third, as the number of missions increases, the number of 
trained and experienced flight support personnel cannot keep up 
with the extreme demands caused by information overload. Automated 
aids and operator assistance allow for productivity enhancement and 
maintaining the required level of flight support. [Hansen 1988] 


To automate the flight and instrument integration and control 
process, the Laboratory has directed a major effort toward the 
incorporation of artificial intelligence into spacecraft sensor 
and space vehicle integration and test, and flight operations 
areas. 


This effort has led to the development of several knowledge- 
based systems to improve the JPL spacecraft and instrument command 
and control process. Specifically these expert systems are: 
SHARP, the Spacecraft Health Automated Reasoning Prototype, is one 
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of a number of Mission Operations Productivity Enhancement Program 
(MOPEP) activities. SME, Spacecraft Monitoring Environment, is 
being incorporated into the satellite integration and test domain. 
EPDM, the Electrical Power Data Monitor, is being designed to 
monitor the Voyager spacecraft power systems, and DAM, Data 
Analysis Module. 


The Spacecraft Health Automated Reasoning Prototype (SHARP) 

The Spacecraft Health Automated Reasoning Prototype has been 
designed to incorporate the experience of the lead Voyager 
spacecraft telecommunications engineer into a useable 
knowledge base. Data from this knowledge source is 
assimilated in a knowledge base that will be used as an 
around-the-clock mission operations assistant in support of 
the Voyager spacecraft's upcoming Neptune Encounter. The LISP 
expert system shell was used in the development. Data Views 
provides the graphical interface. SHARP is implemented on a 
Symbolics 3670. 


Spacecraft Monitoring Environment (SME) 

The Spacecraft Monitoring Environment is being developed to 
aid in the Galileo spacecraft integration process. The SME 
will provide a real time autonomous spacecraft test sequencing 
and data monitoring of integration and test activity. SME 
resides on a Sun 386/i. Data Views is being used to provide 
high-level contextual graphic displays and windowing 
capability. 


Electrical Power Data Monitor (EPDM) 

The Electrical Power Data Monitor is currently under 
conceptual design. The EPDM will be designed to oversee the 
Voyager spacecraft power systems during the Neptune Encounter. 
The C Language Integrated Production Systems (CLIPS) developed 
by Johnson Space Center Mission Planning and Analysis 
Division's Artificial Intelligence Section, will be used as 
the expert system shell. The EPDM development will be on a Sun 
3/260. 


Generic Payload Operations Control Center (GPOCC) 

The Generic Payload Operations Control Center is a concept 
being developed to apply automation to instrument and 
satellite testing, as well as a mission operations 
environment. The GPOCC will couple expert systems with high 
level contextual graphical data displays for ease of user 
interpretation. A modest prototype was developed on a Apple 
Macintosh II to demonstrate user interfaces and functionality 
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of the GPOCC concept. 


The Generic POCC will apply expert system technology to four 
areas: 1) mission flight planning for science instrument and flight 
sequence planning, and command constraint checking; 2) control of 
on-board data storage devices, memory management, and memory 
comparison; 3) DSN 26-meter subnet and TDRSS telecommunications 
link scheduling; 4) satellite telemetry data monitoring, trend 
Analysis, prediction forecast, anomaly detection, fault isolation, 
diagnosis, and corrective action strategy. fJPL D-5435 1988 J 


The Generic Payload Operations Control Center effort is intended 
to support the arduous transitions between development, instrument 
and satellite integration and test, and flight operations. 


However, lack of a funding source and development manpower 
restrictions has permitted only piecemeal development. For this 
reason, an implementation strategy was generated which provided for 
a phased progression of development. The Data Analysis Module was 
selected for initial implementation. DAM builds on the experience 
gained in development of telecommunications and power systems, and 
spacecraft integration and test systems. However, these systems 
were designed to be operated within a restricted domain of their 
own unique environment (i.e. telecommunications, powers systems) . 
DAM is intended to develop an expert system that encompassed 
operation of all the subsystems of a complete spacecraft. 

Other NASA Center Expert System Implementations 

Several other NASA space centers have developed expert systems 
for use in a mission operations environment. All of the systems 
have been well thought-out and impressively implemented. The 
Goddard Space Flight Center has developed two such systems. The 
Communications Link Expert System (CLEAR) for Cosmic Background 
Explorer (COBE) developed by the GSFC Data Systems Applications 
Branch of the Data Systems Technology Division, [Hughes 1987] and 
MOORE (named after Mr. Bob Moore at TRW who was the knowledge 
source) a prototype expert system for diagnosing TDRSS spacecraft 
attitude control problems, developed by Westinghouse Electric for 
the GSFC. [Howlin 1988] 


The Johnson Space Center Mission Planning and Analysis Division 
has developed an expert system called INCO, for the Instrumentation 
and Communications Officer prototype designed to monitor and 
analyze Space Shuttle telecommunications links. [Madison 1988] 
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Data Analysis Module 

The Data Analysis Module will monitor telemetry data and perform 
continual data monitoring and trend analysis based on its knowledge 
base and historic data archived on an optical disk storage device. 
Because of cost constraints, an off-the-shelf hardware PCM 
decommutator will be used to frame synchronize and channelized the 
data stream instead of building a software frame synchronizer. 
The DAM prototype will include a worm-drive optical disk to 
maintain a continuous "knowledge" database of past system 
performance characteristics. 


The Data Analysis Module contains four rule domains. These 
domains are: 

MONITORING - monitoring and interpreting instrument 
and satellite behavior. 

DIAGNOSIS - determine origin of system malfunctions 
inferred from knowledge base. 

PREDICTION - inference of predicted performance based 
on historic performance and current trends. 

RECOMMENDATION - developing and prescribing corrective 
action for diagnosed problems. 


The goal of the Data Analysis Module is to achieve consistent, 
dependable and validatable performance, [Stagner 1988] to demonstrate 
thorough, reliable and fast reasoning, and to reduce sometimes- 
overwhelming loads placed on flight analysis personnel. 


The DAM processing flow is show in figure 1. 


Figure 1 Data Analysis Module Process Flow 
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Input 
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Data Analysis Module Mission Domain 


In implementing the Data Analysis Module, the science and 
mission user domain framed the overall structure and requirements 
of the DAM. DAM will function in an instrument test environment, 
a spacecraft integration and test environment, and lastly, in an 
actual mission operations environment. 


In the instrument test environment the DAM will allow the 
instrument principal investigator or scientist to observe operating 
conditions during development in the same environment and equipment 
configuration as will be used in flight. 


In the spacecraft integration and test environment the DAM must 
provide data-handling flexibility and integrated displays to lend 
itself to the task of releasing satellite integration and test 
personnel from constantly modifying a complex data handling system 
as system requirements change. 


The DAM must support a mission operations environment which will 
be a natural progression from the instrument and satellite 
integration and test environment. The DAM support of the Mission 
Operations System (MOS) will differ little from the prelaunch 
Ground Data Systems testing and MOS training activities. The major 
difference is the addition of "live" flight telemetry data input 
to the input process. This similarity significantly aids in the 
transition from single-instrument integration to full-up on-orbit 
operations. 


The DAM telemetry task will be coupled to a display systems 
design to provide more user-efficient data interpretation by use 
of symbolic representation including pop-up windows, scroll bars, 
graphics, colors representative of data states, and interactive 
icons. 


Data Analysis Module Design Concept 

Knowledge representation is a central issue in the development 
of intelligent systems. While a number of knowledge representation 
formalities and techniques have been developed, most are based on 
semantic networks and frames. [Barr 1981 ;Cho 1985] These may have 
limited application domains which depend on complex deductive 
strategy. 


Expert systems have been developed to provide telemetry data 
analysis. These systems have examined data related to a unique 
spacecraft subsystem (i.e. telecommunications, and power in SHARP 
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and EPDM, respectively) . The DAM examines the input telemetry 
stream on a criticality prioritized basis. 


Early in system design each telemetry channel was assigned a 
criticality level. Once this was done, a relationship for each 
parameter was established. The level two relationship is flowed 
down through to sub-tier prioritized classes. The result was an 
interwoven tree of rule classes based on individual channel 
priority and importance. This entity-relation approach allows 
critical spacecraft and sensor faults to be monitored constantly 
and faults to be quickly detected. 


Each prioritized telemetry channel is assigned a frame slot. 
These slots explicitly point downward to assigned subclasses, and 
up to super classes. This enables DAM frames (a knowledge 
representation scheme that associates features with an object in 
terms of various slots and slot values) to capture hierarchical 
taxonomies. An example of the DAM rule hierarchy is shown in 
figure 2. 


Figure 2 DAM Rule Classes 



Hardware Configuration 

To reduce the cost of the initial implementation the conceptual 
testbed was established on an IBM PC/AT clone using CLIPS as the 
expert system shell. Simulated spacecraft telemetry was used for 
the data source. Formal prototyping will be done in the JPL 
Mission Operations Division's Operations Engineering Laboratory 
(OEL) on a SUN 4/260 using an Advanced Digital Data Systems ADDS- 
100. PCM decommutator to provide conditioning of the serial 
telemetry data stream. 


362 













The initial testbed configuration is: 

System 1800 AT 80286 running at 8 MHZ 

80287 co-processor 

640 Kbyte RAM 

2.5 Mbyte Extended RAM 

with 80 Mbyte hard disk and 30 Mbyte auxiliary storage 

1.2 Mbyte floppy disk drive 

60 Mbyte streaming tape storage 

VGA display with NEC Multisync II color monitor 

The formal OEL prototype configuration will be: 

Sun 4/260 68020 running at 16 MHZ 
8 Mbyte RAM 

two 71 Mbyte hard disks 
60 Mbyte streaming tape storage 
1152 x 900 bit-mapped color display 

Development Approach 

The DAM will be implemented in two stages, a conceptual testbed, 
and a formal prototype. To validate the conceptual design of the 
Data Analysis Module a testbed consisting of approximately twenty 
five rules for the five spacecraft subsystems. Following the 
testbed demonstration a formal prototype will be implemented. The 
prototype will consist of roughly one hundred twenty five rules and 
will be configured as a flight support workstation. 


The DAM testbed will be use to develop the initial telemetry 
channel hierarchial relations. The testbed will demonstrate our 
understanding of the telemetry problem domain. This initial 
testbed model will be a proof of concept. Once the testbed has 
shown our comprehension of the domain, prototype design will begin. 
Testbed development is expected to be completed by the end of the 
year. The conceptual design of the testbed is shown in figure 3. 

Figure 3 DAM Testbed Conceptual Design 
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Both the testbed and the formal prototype will use CLIPS as the 
expert system shell. CLIPS was chosen for several key features: 
these include forward rule chaining, rule syntax, multi-field 
functions, embeddable in C programs, window and mouse interfaces, 
portability (testbed to prototype) , tool set (rule break pointing, 
fact address comparisons, style checking and cross-referencing) , 
and complete documentation. 


The prototype will allow evaluation of the DAM hardware and 
software performance, as well as ferret out problems employing 
actual telemetry data sets. The formal prototype development will 
be completed by the end of 1989. 


The monitoring, diagnosis, and recommendation rules are combined 
into a single task. The predictive model is generated from past 
and current data activity. The DAM will also log all expert system 
activity for later analysis. 


System Architecture 

The basic testbed architecture incorporates three components: 
the CLIPS expert system shell, the input data source, and the 
output user displays. 


The ADDS-100 telemetry decom provides the DAM with simulated 
communications links from the Tracking and Data Relay Satellite 
System (TDRSS) , White Sands Ground Terminal (WSGT) and the JPL Deep 
Space Network (DSN) 26-meter subnet. This telemetry source data 
is frame synchronized, and decommutated by the PCM hardware 
decommutator. The channelized data is archived. 

The DAM expert system provides telemetry data monitoring and 
interpreting, and fault isolation. The DAM will then determine 
origin of system malfunctions inferred from the knowledge base and 
develop and prescribe corrective actions for the diagnosed 
problems. 


The output user displays inform the user of current status and 
alert the user to any data irregularities. 


Data Monitor 


The Data Analysis Module monitors data values to detect faults. 
These faults are identified, a diagnosis is performed, and 
corrective action strategy proposed. As the DAM examines the input 
serial telemetry stream each data channel is placed in a predefined 
frame slot and compared against initial user defined expected data 
boundary value conditions. If a value exceeds the established 
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boundary value the rules associated with that data channel fire. 
If a nominal value is present the current value is compared against 
previous data values. 


Fault Diagnosis and Corrective Action Recommendation 

When channel values exceed boundary conditions faults are 
isolated and corrective action recommendations are predicted by the 
compound rules classes governed by assertive clauses. For example: 

(assert (RWA run-away 

ACS Voltage High 
SCI Voltage Low 
S/A Voltage Low 
RWA PWR On) ) 

Predictive Modeling 

While the DAM is performing continual data monitoring trend 
analysis based on historic data is preformed. Trend variations are 
examined for fault conditions, inference of predicted performance 
is made based on historic values and recommendations are made for 
corrective action. The system maintains a continuous "knowledge" 
base of past telemetry system performance characteristics. 


Knowledge Representation 

The DAM man-machine interface will be via easily interpreted 
contextual graphic displays. These interactive video displays will 
provide a direct representation of the intrinsic images associated 
with the instrument and satellite telemetry and telecommunications 
systems. Multiple display screens with pop-up windows and high 
resolution graphics will be linked through context and mouse- 
sensitive icons and text. 


Implementation 

The DAM design will insure that implementation meets the 
requirements specified in JPL software standards as well as 
software system integration and test standards. [JPL D-4000; D-5000 JPL 

1988] 

Summary 

Based on the knowledge gained in the design and development of 
the Data Analysis Module, the Generic Payload Operations Control 
Center will provide for consistent, dependable and validatable 
performance. It will demonstrate reliable reasoning, and reduce 
the requirement for a large integration and test, as well as a 
mission flight support staff. 
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